Companies often upgrade or transition between database servers and systems to improve database response times or otherwise improve their database performance. Transferring the data from the old database system to the new database system is a time intensive and labor intensive task. In other circumstances, data from multiple databases with different schemas need to be merged.
A mapping between the two databases is created to enable a reorganization of the data in the old database to fit the organization of the new database. The new database may have different methods for maintaining or selecting primary and foreign keys, organizing and storing tables and other differences from the old database. The data from each record in the old database is read from the old database and then inserted into the new database according to the mapping.
The transfer of data is performed by a program in communication with both the old database and the new database. This data transfer program starts a transaction with each database and reads each record from the old database and maps that record to the new database in a single transaction. The execution of this transaction can take a long time depending on the size of the old database. As a result, the old and new databases are inaccessible during the transaction and other programs are not able to access or utilize the old or new database. Further, the transaction cannot be easily interrupted or stopped. Such occurrences can disrupt the transfer process and corrupt it. Transactions must be completed in their entirety or rolled back. Thus, an interrupt of a large transaction forces a large rollback and then a complete restart to the transaction.